Robust artificial neural networks and outlier detection. Technical report

Large outliers break down linear and nonlinear regression models. Robust regression methods allow one to filter out the outliers when building a model. By replacing the traditional least squares criterion with the least trimmed squares criterion, in which half of data is treated as potential outliers, one can fit accurate regression models to strongly contaminated data. High-breakdown methods have become very well established in linear regression, but have started being applied for non-linear regression only recently. In this work, we examine the problem of fitting artificial neural networks to contaminated data using least trimmed squares criterion. We introduce a penalized least trimmed squares criterion which prevents unnecessary removal of valid data. Training of ANNs leads to a challenging non-smooth global optimization problem. We compare the efficiency of several derivative-free optimization methods in solving it, and show that our approach identifies the outliers correctly when ANNs are used for nonlinear regression

Foothill: A Quasiconvex Regularization for Edge Computing of Deep Neural Networks

Deep neural networks (DNNs) have demonstrated success for many supervised learning tasks, ranging from voice recognition, object detection, to image classification. However, their increasing complexity might yield poor generalization error that make them hard to be deployed on edge devices. Quantization is an effective approach to compress DNNs in order to meet these constraints. Using a quasiconvex base function in order to construct a binary quantizer helps training binary neural networks (BNNs) and adding noise to the input data or using a concrete regularization function helps to improve generalization error. Here we introduce foothill function, an infinitely differentiable quasiconvex function. This regularizer is flexible enough to deform towards $L_1$ and $L_2$ penalties. Foothill can be used as a binary quantizer, as a regularizer, or as a loss. In particular, we show this regularizer reduces the accuracy gap between BNNs and their full-precision counterpart for image classification on ImageNet.Comment: Accepted in 16th International Conference of Image Analysis and Recognition (ICIAR 2019

Bounded Influence Regression in the Presence of Heteroskedasticity of Unknown Form

In a regression model with conditional heteroskedasticity of unknown form, we propose a general class of M-estimators scaled by nonparametric estimates of the conditional standard deviations of the dependent variable. We give regularity conditions under which these estimators are asymptotically equivalent to M-estimators scaled by the true conditional standard deviations. The practical performance of these estimators is investigated through a Monte Carlo experiment

Functional significance may underlie the taxonomic utility of single amino acid substitutions in conserved proteins

We hypothesized that some amino acid substitutions in conserved proteins that are strongly fixed by critical functional roles would show lineage-specific distributions. As an example of an archetypal conserved eukaryotic protein we considered the active site of ß-tubulin. Our analysis identified one amino acid substitution—ß-tubulin F224—which was highly lineage specific. Investigation of ß-tubulin for other phylogenetically restricted amino acids identified several with apparent specificity for well-defined phylogenetic groups. Intriguingly, none showed specificity for “supergroups” other than the unikonts. To understand why, we analysed the ß-tubulin Neighbor-Net and demonstrated a fundamental division between core ß-tubulins (plant-like) and divergent ß-tubulins (animal and fungal). F224 was almost completely restricted to the core ß-tubulins, while divergent ß-tubulins possessed Y224. Thus, our specific example offers insight into the restrictions associated with the co-evolution of ß-tubulin during the radiation of eukaryotes, underlining a fundamental dichotomy between F-type, core ß-tubulins and Y-type, divergent ß-tubulins. More broadly our study provides proof of principle for the taxonomic utility of critical amino acids in the active sites of conserved proteins

A habituation account of change detection in same/different judgments

We investigated the basis of change detection in a short-term priming task. In two experiments, participants were asked to indicate whether or not a target word was the same as a previously presented cue. Data from an experiment measuring magnetoencephalography failed to find different patterns for “same” and “different” responses, consistent with the claim that both arise from a common neural source, with response magnitude defining the difference between immediate novelty versus familiarity. In a behavioral experiment, we tested and confirmed the predictions of a habituation account of these judgments by comparing conditions in which the target, the cue, or neither was primed by its presentation in the previous trial. As predicted, cue-primed trials had faster response times, and target-primed trials had slower response times relative to the neither-primed baseline. These results were obtained irrespective of response repetition and stimulus–response contingencies. The behavioral and brain activity data support the view that detection of change drives performance in these tasks and that the underlying mechanism is neuronal habituation

A comparison of A-level performance in economics and business studies: how much more difficult is economics?

This paper uses ALIS data to compare academic performance in two subjects often viewed as relatively close substitutes for one another at A-level. The important role of GCSE achievement is confirmed for both subjects. There is evidence of strong gender effects and variation in outcomes across Examination Boards. A counterfactual exercise suggests that if the sample of Business Studies candidates had studied Economics nearly 40% of those who obtained a grade C or better in the former subject would not have done so in the latter. The opposite exercise uggests that 12% more Economics candidates would have achieved a grade C or better if they had taken Business Studies. In order to render a Business Studies A-level grade comparable to an Economics one in terms of relative difficulty, we estimate that a downward adjustment of 1.5 UCAS points should be applied to the former subject. This adjustment is lower than that suggested by correction factors based on conventional subject pair analysis for these two subjects

Integrating Multiple Sources of Knowledge for the Intelligent Detection of Anomalous Sensory Data in a Mobile Robot

For service robots to expand in everyday scenarios they must be able to identify and manage abnormal situations intelligently. In this paper we work at a basic sensor level, by dealing with raw data produced by diverse devices subjected to some negative circumstances such as adverse environmental conditions or difficult to perceive objects. We have implemented a probabilistic Bayesian inference process for deducing whether the sensors are working nominally or not, which abnormal situation occurs, and even to correct their data. Our inference system works by integrating in a rigorous and homogeneous mathematical framework multiple sources and modalities of knowledge: human expert, external information systems, application-specific and temporal. The results on a real service robot navigating in a structured mixed indoor-outdoor environment demonstrate good detection capabilities and set a promising basis for improving robustness and safety in many common service tasks.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

In situ kinetic measurements of α-synuclein aggregation reveal large population of short-lived oligomers.

Knowledge of the mechanisms of assembly of amyloid proteins into aggregates is of central importance in building an understanding of neurodegenerative disease. Given that oligomeric intermediates formed during the aggregation reaction are believed to be the major toxic species, methods to track such intermediates are clearly needed. Here we present a method, electron paramagnetic resonance (EPR), by which the amount of intermediates can be measured over the course of the aggregation, directly in the reacting solution, without the need for separation. We use this approach to investigate the aggregation of α-synuclein (αS), a synaptic protein implicated in Parkinson's disease and find a large population of oligomeric species. Our results show that these are primary oligomers, formed directly from monomeric species, rather than oligomers formed by secondary nucleation processes, and that they are short-lived, the majority of them dissociates rather than converts to fibrils. As demonstrated here, EPR offers the means to detect such short-lived intermediate species directly in situ. As it relies only on the change in size of the detected species, it will be applicable to a wide range of self-assembling systems, making accessible the kinetics of intermediates and thus allowing the determination of their rates of formation and conversion, key processes in the self-assembly reaction

lp-Recovery of the Most Significant Subspace among Multiple Subspaces with Outliers

We assume data sampled from a mixture of d-dimensional linear subspaces with spherically symmetric distributions within each subspace and an additional outlier component with spherically symmetric distribution within the ambient space (for simplicity we may assume that all distributions are uniform on their corresponding unit spheres). We also assume mixture weights for the different components. We say that one of the underlying subspaces of the model is most significant if its mixture weight is higher than the sum of the mixture weights of all other subspaces. We study the recovery of the most significant subspace by minimizing the lp-averaged distances of data points from d-dimensional subspaces, where p>0. Unlike other lp minimization problems, this minimization is non-convex for all p>0 and thus requires different methods for its analysis. We show that if 0<p<=1, then for any fraction of outliers the most significant subspace can be recovered by lp minimization with overwhelming probability (which depends on the generating distribution and its parameters). We show that when adding small noise around the underlying subspaces the most significant subspace can be nearly recovered by lp minimization for any 0<p<=1 with an error proportional to the noise level. On the other hand, if p>1 and there is more than one underlying subspace, then with overwhelming probability the most significant subspace cannot be recovered or nearly recovered. This last result does not require spherically symmetric outliers.Comment: This is a revised version of the part of 1002.1994 that deals with single subspace recovery. V3: Improved estimates (in particular for Lemma 3.1 and for estimates relying on it), asymptotic dependence of probabilities and constants on D and d and further clarifications; for simplicity it assumes uniform distributions on spheres. V4: minor revision for the published versio